Skip to content

fix(ci): run the JS examples with vitest - #925

Merged
ParidelPooya merged 2 commits into
mainfrom
fix/js-examples-vitest
Nov 10, 2026
Merged

ParidelPooya merged 2 commits into
mainfrom
fix/js-examples-vitest

Conversation

@ParidelPooya

Copy link
Copy Markdown
Contributor

Issue #, if available

None. The js-examples job has failed on main since 2026-11-09. Its last pass was on 2026-11-02.

Description of changes

What

The JS examples harness (.github/scripts/js_examples/run.py) now runs the JS SDK's example tests with vitest. It still runs jest for a JS ref that predates the move to vitest.

Why

  1. The harness ran the examples with npx --no-install jest and read jest's JSON results.
  2. test: replace jest with vitest in all packages aws-durable-execution-sdk-js#1167 replaced jest with vitest in every JS package, on 2026-11-03.
  3. The examples package now has "test:integration": "NODE_ENV=integration vitest run" and the devDependency vitest ^5.1.0. It has no jest package and no jest.config.integration.js.
  4. So npx stopped with npx canceled due to missing packages ... ["jest@30.5.2"].
  5. So jest wrote no results file, and the harness exited with jest wrote no results to .../jest.json.

Change

  • detect_test_runner() returns jest when the examples package has jest.config.integration.js. Otherwise it returns vitest.
  • runner_command() builds the command for the detected runner. The jest command is unchanged.
  • The harness sets NODE_OPTIONS=--experimental-vm-modules only for jest. vitest does not need it.
  • --test-args passes extra arguments to the runner. The old name, --jest-args, still works.
  • The first results file is named after the runner: vitest.json, or jest.json for an old ref. Retry files keep the name retryN.json.
  • Log lines and comments now name the test runner, not jest. The README describes vitest and the runner detection.

The jest flags map to vitest as follows.

jest (old) vitest (new) Notes
--config jest.config.integration.js none vitest loads vitest.config.ts. That config reads NODE_ENV=integration, which the harness already sets.
config testMatch: src/examples/**/*.test.ts config include: src/examples/**/*.test.ts Set by the JS config in integration mode.
config testNamePattern: "cloud" config testNamePattern: "cloud" Set by the JS config. A file with no cloud test still reports as "no cloud tests".
config testTimeout: 120000 config testTimeout: 120000 Set by the JS config.
--bail=0 over config bail: true --bail=0 over config bail: 1 The harness wants every result.
--maxWorkers=N --maxWorkers=N The JS config sets maxWorkers: 16 when CI is set. The CLI value overrides it. The JS config also sets pool: "threads".
--json --outputFile=F --reporter=default --reporter=json --outputFile.json=F The default reporter keeps the progress output on the console.
--runTestsByPath P... positional P... See the assumption about path filters below.
NODE_OPTIONS=--experimental-vm-modules not set Only jest's CommonJS runtime needs it for dynamic import().
setup files vitest.setup.ts from the config It deletes _X_AMZN_TRACE_ID. The harness passes nothing.

The environment the tests read is unchanged: NODE_ENV=integration, LAMBDA_ENDPOINT and FUNCTION_NAME_MAP. Test selection, the localOnly and otel/ exclusions, the invoke proxy, the Lambda shim, retries, report.json and the exit codes are unchanged. The report.json key jest_exit_errors keeps its name, so readers of the report need no change.

Old JS refs

The workflow has a js_ref input, and run.py has --js-ref and --js-dir. So a run can build a JS commit from before aws/aws-durable-execution-sdk-js#1167. Such a commit has jest.config.integration.js and runs only with jest. Supporting both runners costs one file check and one if in the command builder. The jest command is the existing code, moved into a function. So the harness supports both runners. The README section "A failure caused by a JS change" relies on runs against older JS commits, so this keeps that procedure working across the change.

Design

  • Design note (rule 3): not needed. The change keeps the harness's options and adds the alias --test-args.
  • Simpler options considered (rule 2): support only vitest. That removes about 25 lines. A --js-ref run against a JS commit before 2026-11-03 would then fail. Runs against an older JS commit are the README's procedure to tell a Python regression from a JS change. So the PR keeps jest support.
  • Root cause (rule 5): the JS repository removed jest from the examples package, and the harness hard-coded npx jest.
  • Net change in source lines and in public API members: run.py +119/-50, README +17/-11, invoke_proxy.py +1/-1, tests +105/-3. No public API change. The harness gains the --test-args option.

Assumptions

  • vitest's json reporter writes the fields that outcomes() reads: testResults[].name (the absolute file path), testResults[].status (passed or failed) and assertionResults[].status. Evidence: the JsonReporter class in vitest 5.1.0, the version in the JS lockfile. A skipped test has the status skipped. jest used pending. outcomes() treats any status other than passed or failed as skipped, so both work.
  • vitest treats each positional argument as a case-insensitive substring of the test file path relative to the root. Evidence: filterTestFiles() in vitest 5.1.0. Every selected path starts with src/examples/. So a selected path can also match another file only if that file's path contains the whole selected path. No examples path does that today. The full run below reported exactly the 115 selected files.
  • --bail=0 on the command line overrides bail: 1 in the config. Evidence: a run with --test-args='--testTimeout=1' on the 14 wait-for-callback/ files. All 14 files ran, and all 17 tests failed, in both the first run and the retry. With bail: 1, vitest would have stopped the run after the first failed test.

Decisions not covered by the issue or the design note

  • --test-args is the new option name, because the arguments now go to vitest. --jest-args stays as an alias, so existing commands keep working.
  • The first results file is named after the runner (vitest.json), because the file holds that runner's output.

Testing

  • Full harness run against JS main at ca59e68, as CI runs it: hatch run dev-testing:js-examples --js-dir <js checkout> --out <dir>, with CI=true, hatch 1.16.5 and Node.js 24. Result: 115 test files selected, 112 passed (0 on retry), 0 failed, 3 have no cloud tests, 0 not run. 6 localOnly files were skipped. vitest reported 136 tests passed and 10 skipped, in 119 s. Exit status 0.
  • Failure path against JS main: --test-args='--testTimeout=1' wait-for-callback/. Result: 14 files selected, 14 failed, then retried, then reported as FAIL. Exit status 1.
  • Old JS ref: a run of step/ against JS commit 21074ed, the parent of the vitest change. The harness detected jest and wrote jest.json. Result: 9 files selected, 7 passed, 0 failed, 2 have no cloud tests. Exit status 0.
  • Unit tests: hatch run dev-testing:python -m pytest .github/scripts/tests/test_js_examples.py: 70 passed. New tests cover the runner detection, the vitest and jest commands, the --test-args and --jest-args options, and parsing of a vitest json result.
  • hatch fmt --check (hatch 1.16.5, ruff 0.13.2) on the 3 changed Python files: passed.
  • Edited test expectation: test_jest_args_are_split became test_test_args_are_split. It now reads args.test_args, because the option's destination changed. It runs once with --test-args and once with --jest-args.

Checklist

  • Every new branch has a test that fails without it (rule 4)
  • For a bug fix: the first commit adds a test that fails on the current code (rule 5). The fix and its tests are in one commit. The new tests call runner_command() and detect_test_runner(), which the current code does not have.
  • Every edited test expectation is listed in the description, with the reason (rule 5)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

The JS SDK replaced jest with vitest in its examples package. So
npx jest found no jest package, and the js-examples job failed.

The harness now runs vitest when the examples package has no
jest.config.integration.js. It still runs jest for an older JS ref
that has that config. Both runners write the same JSON fields, so
result parsing and reporting do not change.

--test-args replaces --jest-args. --jest-args still works.
@ParidelPooya
ParidelPooya marked this pull request as ready for review November 10, 2026 18:52
@ParidelPooya
ParidelPooya deployed to ai-pr-review-runtime November 10, 2026 18:52 — with GitHub Actions Active
@ParidelPooya
ParidelPooya deployed to ai-pr-review-runtime November 10, 2026 18:53 — with GitHub Actions Active
Comment thread .github/scripts/js_examples/run.py Resolved
Comment thread .github/scripts/js_examples/run.py Resolved
@github-actions

This comment has been minimized.

hln33
hln33 previously approved these changes Nov 10, 2026
A test now checks that only jest gets --experimental-vm-modules in
NODE_OPTIONS. It fails when the runner check is inverted.

The new --test-args option is removed. The harness keeps --jest-args,
which now passes arguments to whichever runner it picks. So the PR
adds no CLI option.
@github-actions

Copy link
Copy Markdown
Contributor

Codex AI review

One prior P2 approach finding remains at .github/scripts/tests/test_js_examples.py:395. No additional correctness issues confirmed. Tests were not executed during this read-only review.

Suppressed repeats

  • arf_v1_kcmyhqpu7kcbjm6vdd5xiscu4u — deferred (maintainer verdict)

Reviewed commit 38899e9bca1c5feffa5786861568bbce37429304. Workflow run

@ParidelPooya
ParidelPooya merged commit 07a3b4a into main Nov 10, 2026
52 checks passed
@ParidelPooya
ParidelPooya deleted the fix/js-examples-vitest branch November 10, 2026 19:18
@zhongkechen zhongkechen linked an issue Nov 10, 2026 that may be closed by this pull request

This branch was successfully deployed

1 active deployment
ai-pr-review-runtime — 38899e9b Deployed Nov 10, 2026 by ParidelPooya via ai-pr-review / Codex review / Generate Codex review #3036
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: JS examples CI still invokes Jest after Vitest migration

2 participants